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p METHOD AND APPARATUS FOR DETERMINING THE 
SENSITIVITY OF INPUTS TO A NEURAL NETWORK 
ON OUTPUT PARAMETERS " 



TECHNICAL FIELD OF THE INVENTION 

The present invention pertains in general to neural networks, and more 
particularly, to analyzing the operation of a neural network as a function of the sensitivity 
of input parameters on the neural network. 

CROSS-REFERENCE TO RELATED APPLICATIONS 

This application is a continuation-in-part application of U. S. Patent Application 
Serial No. 025,184, filed March 2, 1993, and entitled "Method and Apparatus for 
Analyzing a Neural Network Within Desired Operating Parameter Constraints". 
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BACKGROUND OF THE INVENTION 

Neural networks have been utilized in an ever increasing manner to predict 
system operation in the future such that adjustments to the system can be anticipated and 
also to provide control inputs to a manufacturing control system. These networks 
5 provide a non-linear representation of a plant, which non-linear representation was 

learned through the use of historical training data. Once the system is running, it is often 
desirable to change any of the operating parameters of the system through the use of 
either the control operation of the neural network or to determine how' changes in the 
inputs to the neural network will affect the predicted output. These changes are often 
1 0 constrained by physical limitations of the plant, or by user-defined constraints supplied to 
achieve desired behavior. The present invention provides a novel mechanism for 
achieving such desired behavior while simultaneously satisfying constraints. 

When utilizing neural networks as predictive tools, a system designer may be 
locked into the overall operating system. For example, control networks typically 

1 5 receive a desired output and then generate control inputs to force the control inputs to a 
state that will yield the desired input by minimizing the error value between a predictive 
output and a desired output. These control networks provide as an output a control 
input to the system, which then responds accordingly. However, the way in which the 
input is applied is not controlled. For example, a user may change the desired output to 

20 increase impurity concentration for a given process. This could entail changing the 

flowrate of two valves, changing the heater control, etc. Unfortunately, the operation of 
the plant during this change to achieve the desired output is unknown, and it is not until 
the entire system has settled down and the desired output has been reached that the 
system is operated as desired 

25 ^ order to alter the inputs to a plant to achieve a desired effect in accordance 

with the suggested control input changes from a control system, it is necessary for the 
operator and/or the control system to determine how much each input value should 
change and also in what the sequence of change should be. One factor that complicates 
the control strategy that is implemented is the sensitivity of the output on each of the 
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input values. For example, the outputs in some plants are very sensitive to certain input 
values and exhibit almost no change as a function of changes in other input values. 
However, this fact is not known until the predictive network that represents the plant is 
completely trained on all input values. Therefore, the network requires all input values ir 
5 order to operate. Of course, the ones of the input values that cause no change in the 
output can be clamped to zero, if these input variables can be discriminated, and then the 
network run to define or to predict new and updated input values. Therefore, in order to 
eliminate the least sensitive input values, it is first necessary to determine which of these 
input values causes little or no change in the output and then eliminate these values from 
10 the input. One problem that exists in determining the sensitivity of the input variables on 
the network, and subsequently eliminating the input variables as inputs to the network, is 
whether the dependence of the outputs on the input variables is linear or non-linear. 
Further, it is necessary to determine whether the output is sensitive to a combination of 
input variables as opposed to being sensitive only to individual input variables. For 
1 5 example, a given input variable in isolation may cause no change in the output. 

However, this input variable in combination with a change in another input variable may 
cause a large change in the Output variable. Therefore, non-linear relationships between 
the input variable and the output variable must be discerned in order to accurately 
determine the sensitivity of the output variables on input variables. 
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SUMMARY OF THE INVENTION 



The present invention disclosed and claimed herein comprises a network training 
system for reducing the number of inputs in a non-linear predictive network having 
stored therein a representation of a plant. A dataset is provided consisting of training 
patterns representing the input vanables and measured output vanables associated with 
the network, with a given pattern having input values associated with the input variables 
and corresponding output values associated with the measured output variables. A 
sensitivity processor is operable to determine the sensitivities of each of the output 
vanables as a function of each of the input variables. The sensitivity is the effect of each 
of the input variables on a learned representation of the output variables, which learned 
representation is learned over all of the input variables. Th,s allows the comb.ned effects 
of all other input variables to be taken into account. Once the sensitivity is determined, it 
is compared with predetermined critena and select ones of the input variables chosen ' 
from the dataset. A network model is then trained only on the select input variables and 
1 5 the associated portion of the training patterns, with the model receiving orJ> me selected 
input variables. The model then provides a predictive output for the received ones of the 
selected input variables. The network model is trained in accordance with a 
predetermined training algorithm 



10 



20 



25 



In another embodiment of the present invention, a second network model is 
provided wh,ch is operable to receive as inputs all of the input vanables and provide 
predictive outputs assoc.ated with each of the output variables in the dataset. The 
second network model is trained on all of the input vanables and output vanables in the 
dataset to prov.de the learned representation of the output variables. The effect of each 
of the input variables to the network can then be determined on the learned 
representation of the output variables, with the network allowing the combined effects of 
all other input vanables to be taken into account for determining the effect of any of the 
input variables on any one of the predicted output variables. 

In yet another aspect of the present invention, a single neural network is utilized 
that has an input layer having a defined number of inputs for receiving input variables, an 
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output layer for outputting a predetermined number of outputs and a hidden layer for 
mapping the input layer to the output layer. The neural network is configurable such 
that, in one mode, it can receive on the input layer all of the input variables and be 
trained on the entire dataset. In a second mode it can be configured to receive only the 
5 select ones of the input variables on the input layer and be trained onJy on the select 
portion of the dataset associated with the select ones of the input variables. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

For a more complete understanding of the present invention and the advantages 
thereof, reference is now made to the following description taken in conjunction with the 
accompanying Drawings in which: 

FIGURE 1 illustrates a block diagram of the overall system to provide the 
analyzer function; 

FIGURE la illustrates a detailed diagram of a conventional neural network; 
FIGURE 2 illustrates a detail of the analyzer; 

FIGURE 3 illustrates a detailed block diagram of the control network; 
FIGURE 3a illustrates a block diagram of the iterate block in FIGURE 3; 
FIGURE 4 illustrates a flowchart for the operation of the analyzer at the top 

level; 

FIGURE 5 illustrates a flowchart for generating the display of predicted versus 
actual output; 

FIGURES 6a, 6b and 6c illustrate plots of average total sensitivity, absolute 
average total sensitivity and peak sensitivity versus rank; 

FIGURES 7a, 7b, 7c and 7d illustrate plots of various examples of X-Y plots of 
an input and an output variable over the extent of their range, 

FIGURE 8 illustrates' a plot of sensitivity versus output for select outputs; 

FIGURE 8a illustrates a plot of sensitivity versus percentage for select outputs; 
FIGURE 9 illustrates a flowchart for determining the sensitivity, 

FIGURE 10 illustrates a flowchart for displaying setpoints and "what ifs"; 

FIGURE 1 1 illustrates a flowchart for changing the predict-outputs mode of 
setpoints and "what ifs"; 

FIGURE 12 illustrates a flowchart for changing input parameters; 
FIGURE 13 illustrates a flowchart for predicting the inputs; 
FIGURE 14 illustrates a flowchart for changing the output parameters; 
FIGURE 15 illustrates plots of input and output variables with different cost- 
constraints; 

FIGURE 16 illustrates a plot of the fuzzy-constraint function; and 
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FIGURE 1 7 illustrates a block diagram of the system for training a neural 
network to reduce the number of inputs in accordance with determined sensitivities of 
the outputs on the input variables; 

FIGURE 18 illustrates a flow chart for training a neural network after a 
5 determination of the sensitivities; 

FIGURE 19 illustrates a block diagram of a neural network having inputs with 
different delays; and 

FIGURE 20 illustrates a block diagram of the control network utilizing the 
determined sensitivities to change the predicted inputs to the DCS. 
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DETAILED DESCRIPTION OF THE INVENTION 

Referring now to FIGURE 1, there is illustrated a block diagram of the overall 
system and analyzer. In general, a plant 10 is provided that can be any type of physical, 
chemical, biological, electronic or economic process with inputs and outputs. The plant 
has an output y(t) and control inputs x(t), the control inputs x(t) provided on an input 
12 In addition, the plant 10 has external inputs E(t), which comprise such things as the 
ambient temperature, the humidity, etc These are typically parameters that cannot be 
controlled. The plant also has associated therewith measured state variables s(t). such as 
flowrates, temperature measurements, etc. These are typically measured variables. It 
should be understood that the flowrate or the temperature may be directly associated 
with one of the control inputs x(t) such as, for example, flowrate. Typically, a valve 
constitutes a control input and the flowrate merely represents the setting on that valve. 
Therefore, a setting of the valve would constitute a flowrate. However, it is a measure 
1 5 of this flowrate that constitutes a measured state variables in s(t). 

The control inputs x(t) are generated by a distributed control system 14. The 
output of the distnbuted control system, comprising the control inputs x(t), and the state 
variables s(t), are input to a runtime control network 16, which generates control inputs 
x(t + 1) that are utilized to provide the settings for the distnbuted control system 14. 
20 The runtime control net 1 6, as will be described hereinbelow, incorporates a predictive 
model of the plant 10, in the form of a neural network or any other type of non-linear 
network. An inverse network is also provided for generating the predictive inputs to the 
distributed control system 14. 

The runtime control network 16 operates in accordance with control parameters 
25 stored in a memory block 22. As will be described hereinbelow, the runtime control net 
will operate the distributed control system 14 in accordance with various criteria such as 
a desired target output and desired "cost factors" for the input parameters of the network 
and for the predictive output from the network, and also as to limitations that may be 
placed upon the control inputs, such as rate-of-change and maximum and minimum 
30 constraints on the range of the input values, or combinatorial constraints such as 
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constraints on the ratio or sum of constituents of a mixture. Additionally, the sensitivity 
of the output on certain input variables can be compensated for in order to account for 
these sensitivities. 



A parallel system is provided to the runtime control net 16, which utilizes a 
5 system model 24. The system model 24 is operable to receive on the input thereof either 
training data from a memory 26, or simulated data on a line 28. The output of the 
system model 24 is input to an analyzer 30 which is operable to analyze the overall 
operation of the network. With this configuration,, the plant 10 can be modeled in a 
system model 24, which system model 24 also incorporates the features of the runtime 
1 0 control net 1 6, in order to analyze the operation of the plant as a function of constraints 
that may be placed on the input or on the control inputs to the plant 10, and also on the 
output of the plant, as indicated by the internal prediction of the output. By providing a 
parallel system, this operation can be accomplished completely independent of the 
runtime control net 16. However, once the analysis has been performed by the analyzer 
15 30, new control parameters can be generated, and downloaded to the control parameter 
block 22 for use by the runtime control net 16 in real time. The analyzer is operated 
through use of an input/output device 32 such that an operator can input information to 
the analyzer 30 and this information can be displayed. The analyzer 30 is operable to 
generate the simulated data on the line 28. 

20 Referring now to FIGURE la, there is illustrated a detailed diagram of a 

conventional neural network comprised of input nodes 15, hidden nodes 17 and output 
nodes 18. The input nodes 15 are comprised of N nodes labelled x„ x 2 , ... x N , which are 
operable to receive an input vector x(t) comprised of a plurality of inputs, INPl(t), 
INP2(t), ... INPN(t). Similarly, the output nodes 18 are labelled o„ o 2 , ... o K , which are 
operable to generate an output vector o(t), which is comprised of the output OUTl(t), 
OUT2(t), ... OUTK(t). The input nodes 14 are interconnected with the hidden nodes 17, 
hidden nodes 17 being labelled a„ a 2 , ... a„, through an interconnection network where 
each input node 15 is interconnected with each of the hidden nodes 17 However, some 
interconnection schemes do not require full interconnect. Each of the interconnects has a 
weight W,/. Each of the hidden nodes 1 7 has an output O; with a function g, the output 
of each of the hidden nodes defined as follows: 



25 
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*/ = S(L W l *t + */> 



(1) 



Similarly, the output of each of the hidden nodes 17 is interconnected with substantially 
all of the output nodes 18 through an interconnect network, each of the interconnects 
having a weight W jk 2 associated therewith. The output of each of the output nodes is 
defined as follows: 

°t = g(E*i *j + & (2) 
j-\ 

5 This neural network is then trained to learn the function f(x) that is embedded in the 
neural network from the input space to the output space as examples or input patterns 
are presented to it, and a Total-Sum-Square-Error function is minimized through use of 
a gradient descent on the parameters W jk 2 , W-- 1 , b 1 ], b 2 k. 



f he neural network described above is just one example. Other types of neural 
1 0 networks that may be utilized are those using multiple hidden layers, radial basis 

functions, gaussian bars (as described in U.S. Patent No. 5,1 13,483, issued May 12, 
1992, which- is incorporated herein by reference), and any other type of general neural 
network. In the preferred embodiment, the neural network utilized is of the type referred 
to as a multi-layer perception network 

1 5 Referring now to FIGURE 2, there is illustrated a more detailed diagram of the 

system of FIGURE 1, wherein the overall system model 24 and runtime control net 16 
are multiplexed in operation with the distributed control system 14. The runtime control 
net 16 and system model 24 are each comprised of a predictive model 34 and an inverse 
model 36. The predictive model 34 is represented by the system of FIGURE la, in that 

20 it is operable to receive the control input x(t) and state variables s(t) and output a 
predictive output o p (t), which represents the predictive output of the plant 10. The 
predictive model 34 has therein a stored representation of the plant 10, which stored 
representation is a learned representation which was learned on the training data stored 
in the memory 26. This is a separate training operation that will be described 
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hereinbelow with respect to sensitivities. These are stored or fixed weights which 
determine how the predictive model 34 operates. To the extent that the predictive model 
34 is an accurate model, the actual output of the plant 10 and the predicted output of the 
predictive model 34 will be essentially identical. However, whenever the actual output 
5 of the plant has to be varied, the plant control inputs must also be varied, this effected 
through the runtime control net 16 and the distributed control system 14. The predictive 
model 34 receives the input therefor from the multiplexer 38. The multiplexer 38 is 
operable to receive the simulated control inputs from the line 28, passing through a filter 
46', the actual control inputs to the plant 10 as the variable x(t) and the state variables 
10 s(t) from the distributed control system 14, or the training data from the memory 26. 

With regards to the input, the predictive model 34 can generate a predicted output that is 
a non-linear function of the inputs provided thereto. This predictive output o p (t) is input 
to the analyzer 30. 

In order to provide the control network function, an error is generated so as to 
1 5 minimize the cost in a cost minimization block 42, which is operable to receive the 
• predictive output o p (t) of the predictive model 34 and the inputs to the predictive model 
34. The cost minimization block 42 also receives control parameters from the control 
block 22, which are utilized to calculate an error E, which in the preferred embodiment is 
then processed by the inverse model 36 in accordance with the general operation of a 
20 control network to minimize the cost and generate on an output 44 new control inputs. 
These updated control inputs are input to a block 46 that is labelled "filter". This block 
46 functions to satisfy any "hard" constraints that have been placed on the system before 
they are input to the plant 10, or the predictive model 34. These constraints are of the 
following three types: 1) range constraints; 2) rate-of-change constraints, or 3) 
25 combinatorial constraints. Range constraints are of the form: 

iow*r upptr 

where x/ 0 *" is the lower hard-limit or hard-constraint, and x i oppcr is the corresponding 
upper hard-constraint, meaning that a particular control cannot be 
varied outside of these limits. Rate-of-change constraints are of the form: 
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w/icre: A x = x(l * 1) - x(l) ( 4 ) 

meaning, e.g., that a particular control cannot be changed faster than the prescribed rate. 

Combinatorial constraints are used to satisfy limits on the combinations of 
variables. For example, it is quite common that the sum of all of the flows into a system 
must equal 100% of the flow out. This is known as a mass-balance constraint, and can 
5 be expressed as: 

£ x i 

ieflo* = COnS,an ' (5) 

More generally, we can have any function of the inputs: F(x) = constant, such as the 
ratio of two ingredients that must be constant, i.e., x/xj = Q. 

Additionally, the filter 46 is operable to contain sensitivity parameters. The 
sensitivity parameters, as will be described hereinbelow, are parameters that determine 
how the predicted input values are to be changed prior to inputting them to the DCS 14. 
As will be described hereinbelow, the sensitivity of the output of the plant 10 on the 
various input variables is first determined and then the way in which the inputs are 
applied by the DCS 14 is determined by the sensitivity parameters. For example, one 
input variable may be determined to cause the most change in the output and this input 
1 5 variable will be selected first for application to the input of a plant 10. Further, the 
predicted outputs out of model 36 can be altered as a function of the sensitivity of the 
predicted outputs on line 44. For example, the most sensitive input variables may have 
the predicted values from inverse model 36 passed directly through to the DCS 14 and 
the least sensitive input values may have the predicted values attenuated. This will be 
20 described in more detail hereinbelow. 

The contents of filter 46 are controlled by information received from the control 
parameter block 22. As will be described hereinbelow, the filter 46 is operable to place 
hard constraints on the inputs and/or other constraints such as rate of change, sensitivity 
considerations, etc., that may be required when applying new inputs to the plant 10. As 



10 



WO 94/25933 



PCT/US94/04692 



will be appreciated, the predicted inputs generated by the inverse model 36 are generated 
as a function of the manner in which the overall control net minimizes the error function 
output by the minimization block 42. This will be described in more detail hereinbelow. 

The output of the filter 46 is input to a latch 48, the output of which is input as 
5 the control inputs x(t + 1) to the DCS 14. The latch 48 is operable to only pass through 
new control inputs during the runtime mode. During the analysis mode, the latch 48 
prevents new data from being updated to the DCS 14. The output of filter 46 is also 
input back to the analyzer. The analyzer 30 is operable to control the overall operation 
of the system through either placing it in a runtime mode or placing it in an analysis 

10 mode. In the analysis mode, information is displayed on a display 50 with inputs 

received from an input device 5 1 . Further, a filter 46' is incorporated on the line 28 to 
apply the hard constraints to the simulated inputs. Another filter, filter 47, is 
incorporated on the input to the predictive model 34 to allow constraints to be applied 
directly to the inputs to the model. Both filter 46' and filter 47 are controlled by block 

15 22. 

Referring now to FIGURE 3, there is illustrated a block diagram of a control 
system for optimization/control of a plant's operation in accordance with predetermined 
weights. A plant is generally shown as a block 10 having an input for receiving the 
control inputs x(t) and an output for providing the actual output y(t). A plant predictive 

20 model 54 is developed with a neural network to accurately model the plant 10 to provide 
an output o p (t), which represents the predicted output of plant predictive model 54. The 
inputs to the plant model 54 are the control inputs x(t) and the state variables s(t). For 
purposes of optimization/control, the plant model 54 is deemed to be a relatively 
accurate model of the operation of the plant 10. In an optimization/control procedure, 

25 various generated parameters stored in parameter blocks 49, 51 and 53 are input to the 
cost minimizer 42. The parameters are cost coefficients stored in the parameter block 
49, fuzzy constraints stored in the parameter block 5 1 and desired values stored in the 
block 53. The operation of each of these will be described hereinbelow. These 
parameters from the parameter blocks 49, 51 and 53 are utilized by the cost minimization 

30 block 42, which also receives the output of the plant model o p (t) cost and a mechanism 
for generating new control inputs that satisfy the constraints in filter block 46 so as to 
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of.be predictive tnode, frozen This bach pr„p, g a„„„ of the error throunh ,he network 
> stmtlar to an invers.on of the network wmh the output of the tnverse pla'nt mo del 56 
representing a Ax(« + ., utilized in a gradten, descent operat.on il,us,rat=d by a„ i.era.e 
0 block 57 I„ „ P e ra „„„, as illustrated , n delai| in F[GUR£ Ja [he va|ue a . s ^ 

tntttally to the inpn, value *„, and tins sun, then processed through the fiber 46 and the 
Plant predictive model 54 ro provide a new predictive ourpu. „-(„ and . new OTOr ^ 
tteration continues umi, ,he error is reduced below a prederemnned value The final 
value ,s then oulpu, as the new predictive control variables i(t + l) 

These new x», + „ values compose the control inputs are required ,„ achieve 
the destred operation of the plan, ,0 These are input ,o the plan, control system 14 
wherein a new value is presented ,o the sys,em for mpur an the control variables „,)' 
The control sys,em ,4 is operable ,o receive a generalized co„,ro, i„p u , which can he 
vaned by the dts.nbu.ed control sys.em 14. The general terminology for Ihe back 
propagation of error for con.ro, purposes is "Back Propaga,ion,o-Ac,iva,ion- (BPA) 

In the preferred embodtment, the method utilized ro back propaga.e the error 
■hrough the inverse plan, mode, 56 is to utilize a local gradient descent through ,he 
network from the outpu, to the input w,m the weights frozen The firs, step „ t0 apply 
■ e presen, inputs fo, ho.h rhe contro, vanables a,„ and ,he state variables s( „ i„ I0 , J 
Plan, model 54 to generare the predictive outpu, o W A local gradient descent is then 

mpultmg the error E in accordance wi,h ,he following equation: 
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Ax(r) = ~~t\ 

dx 



(6) 



subject to the constraints F(x), and where E is the error input to the network, described 
in more detail hereinbelow, and where rj is an adjustable "step size" parameter. The 
output is then regenerated from the new x(t), and the gradient descent procedure is 
iterated. In an alternate embodiment, input values of x are chosen at random, subject to 
5 the constraints F(x) and the value of x that minimizes E is the given as the new control 
input. As will also be described hereinbelow, these inputs can be varied as a function of 
the sensitivity- 
Referring now to FIGURE 4, there is illustrated a flowchart for the overall 
operation of the analysis routine by the analyzer 30. The program is initiated at a start 
10 block 66 and then proceeds to a decision block 68 to determine whether the analysis 
operation is to be performed. If not, the program flows back along the "N" path to the 
input of the decision block 68 and, if so, the program flows along a "Y" path to a 
decision block 76 to determine if a sensitivity operation is performed. If so, the program 
flows along a "Y" path to a "Go To" block 78. If the sensitivity operation is not to be 
1 5 performed, the program flows to a decision block along the "N" path to determine if the 
predictive versus actual operation is to be performed. If so, the program flows along a 
"Y" path to a "Go To" function block 82. If not, the program flows along an "N" path to 
a decision block 84 to determine if the Setpoints operation is to be performed. If so, the 
program flows a "Y" path to a "Go To" block 86. If not, the program flows along an 
20 "N" path to a return block 88. 

Predictive versus actual tests are designed to run through data from the plant to 
determine how accurate the model is when it is presented with the training patterns and 
novel testing patterns. Sensitivity is designed to determine which variables are the most 
sensitive for affecting the output variables The Setpoints operation is designed to allow 
25 the user to perform "software designed- experiments" by changing input and output 
parameters of the models, i.e., manipulate the operating conditions of the models. 
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Refenrng now ,„ FIGUR£ 5 , here , s ^ 

opnranon of co mp an„ g ,he predicted o u ,p u , to the actual ou t pu t , „ W eh is Mated a, a 
block 90, and then ,he program flows , 0 , declsl0 „ b|ock „ 2 (o jf fc ^ 

. loaded from memory 26. If so, ,he program flows along an "M" parh ,„ a function 
5 block 9< ,„ ,oad rhe Datase, and, if,, is n0 , ,o be loaded, me program flows through a 
path, bo,h oo, pu , pa, hs flowtng ,„ , ne input of a decision b|ock ^ ^ 
whether the mode, is loaded ,f no,, the program flows an "N" path to a Sanction block 
98 to tad rhe mode, and then ,„ the input of a tocron block 100 „ m „ ^ ^ 
After lhe m „d e , is , 0 aded, the program flows dtrecdv io the Sanction biock ,00 alone a 

path The program then flows ,o a function block 102 to deterafine which of, he 
ourpu,s ,„ ,he ou tpu , vecor „■(„ j, ,„ be alsplayed Whe „ ^ ^ 

to a fianction block ,04 ,„ dis P ,a y ,he P redic,ed o„« P „, versus me acua, ou, pu , and ,he„ 
to the ,„ P u,ofadec,s,o„ block ,0b ,o de,e,mt„e if ,he oner„ion is done If„„, , he 
program flows back ,o ,he ,„ p „, of fianction b,ock ,02 ,„ se,ec, anchor d,s P ,a y a,ong ,he 

P a,b and, ,f no,, ,he program flows along ,he "Y" p a,h ,o a dects,on block ,08 ,o 
detenntne if.be accuracy of ,he pred.crion is sa,isfac,„,y ,f , he accuracy is no, 
sa„sfac,ory, ,h,s tndtca.es ,ha, the mo de, is no, accural, .rained and ,he program wi„ 
•hen flow a,o„g an "N" p„h ,o a fl.nc.ion block „0 ,o perform . mode, ,ra,„„„ 
operation, which is no, desenhed in ,na s disctanre Thc progrm ^ ^ ' 
« lock „2 [f the accuracy is satisfactory, the program wi„ flow d irectly to the Exit 
block 1 ,2 along a "Y" path from decision block ,08 

There are two ,y pes of sensitivity detemnna.ions ,ha, can be perforated 
sens,„vi,v versus rank and senst.ivi.y versus percen, change. A sensitive analysis 
.nd.ca.es bow much efrec, or tnfluence each ,„p„, vanable has on each ou.pu, vanabie 
-5 In ,he P red,c„on mode,, mis mcludes a„ i„ pu , varices However, in a oon.ro, mode, 
only me comrol/exrema, vanables are considered, stnee the ou, P u,s are no, d.recdv ' 
sensmve ,o state vanables As wi„ be descrtbed beretnbeiow, one a.ra.egy fot opumiaing 
a process ,s to idenfrfy ,be , npu , ^ abks ,„ ^ ^ ^ ^ 

then opumiae tbem firs,. ,b,s P rov,di„g the best overall ophmtaauon of the pro oess The 
less .nfluenfial vanables are then ,den,ified and opunuaed in otder ,o amve a, the best 
dynamic control of the process. 
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In the sensitivity versus rank, the sensitivity of the input variables versus the 
output variables is calculated by taking the distribution-averaged partial derivatives of the 
output variables with respect to the inputs. Three separate sensitivity measurements can 
then be computed. These can be the absolute average total sensitivity, the average total 
5 sensitivity and the peak sensitivity. The average is over the distribution of patterns 
specified by the user, and is computed in each of the cases by the following formulas. 
The average sensitivity is as follows: 



Npats 



Average = Tsens t( = y ii 

0 «. _ J dx V'J 



where, N pats is the number of patterns in the Dataset for which the determination is to be 
computed over, and o K . is the i* output for the k* pattern and ^ is the ith input for the 
1 0 kth pattern. Similarly, the average total sensitivity, ATsensy is the sum of the absolute 
values of the partial derivatives as follows: 



Npats £ Q 

Average Absolute = AT sens „ = y i ^ ! 

° ft - 1 i **j : 



(«> 



Finally, the peak sensitivity is the maximum of the partials over all patterns as follows: 



Peak = PkSens, 



'J 



, 1, 2... Npats 



(9) 
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The sensitivities associated with the average, average absolute and peak are collected for 
all of the input/output pairs into a table, and they can be viewed in a plot of the 
sensitivity versus the input variables. It is convenient to sort the variables according to 
rank and to plot the variables so that the most sensitive variables are to the left, the least 
to the right. This sorting is for each view, average, average absolute or peak sensitivity. 
This information, by way of example, is illustrated in Table 1 . 
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TABLE 1 



10 



Rank 


Name 


Time-Delav 


ATsen 


Tsen 


1 


Howl 


-4 


1.40 
0.469 
0.408 
0.349 




2 
3 
4 
5 


press I 
tempi 
flow2 
level 1 


0 
-4 

0 
0 


1.40 
0.469 
0.408 
0.051 


6 


press2 


0 


0.281 
0.178 
0.156 
0.135 
0045 
0.026 


-0.260 


7 


antifoam 


0 
0 


-0.178 


8 


temp3 


-0.080 


9 
10 


press3 
pumpspeed 


0 
0 


-0.051 
0.002 
0.005 



Refernng now to FIGURES 6a, 6b. and 6c, .here are illustrated plots 0 f the 
sens„,vi,y versus rhe rank This provides a dispia, of selec.ed average sensi.ivrty (Tsen, 
average ahsolnt. sensitivity (ATsen), or peak sensiuvtty ( PkS e„). Eaoh iine «ha, is ' 
plotted corresponds ,o a s,„g,e „ utpul variaW< , w „ b four ^ 
The Y-axrs ,s .he calcula.ed sens,,,vi,y values Each plotted poin, on a line represents 
.he output's sensirivtty to one input variable. On each line, the inputs are ordered by a 
magnitude of sensn.vity (of whichever type is being displayed); .has means r„a, i„ put 
vartables do not necessar.l, occur in .he sante order on each line. The X-atris is si mp ,v 
■he rank order To identify ,h. , nput variaWe „ my pmaahr ^ ^ fc ' 
- systent, „ ,s only necessary to move a pointing device onto i, and push and M „ , he 
po,n,,ng dev,ce. ,he vanables name, ,, me delay and se„s,,ivi,v will be shown The - 
value represenring ,he de.ay associa.ed with ,he input vanahie, .his being described _ 
2a herembelow ,f, for exa mpl e, the peak sensirivtty d,agram is displayed, ,be pattern 
number a, which ,he peak occurred (bu, no, the value) will also be shown. 

After ,he sens.,,vi,y analysis has been performed and the senstttvities of the 
outpu. vanables on the input vanables has been deternuned, . son function can be 
perfumed on the input Darase, in order ,o atvange the columns in , he Dataset , such , ha , 
.0 tnpu, variables are .„ the order of their mflueuce on an, one output variable .„ genera, 
sortmg ,s always done „ the order of Average Absolute Sensirivtty, regardless of which 
type of sens,,,.,, is being displayed ,f ,he mode, has only one ou,pur variable sotttng is 
done on an tmmedt.te bas,s Othenvtse, i, is necessary to selec, a given one of the 
output vanables prior ,o sorting and men the Datase. will be sorted according ,o the 
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selected output variable sensitivity to the input variables. As such, the Dataset will then 
be sorted in accordance with the selected output variable. 

If an input variable in the model has multiple time delays associated therewith, it 
is sorted according to the largest sensitivity in any time delay. As will be described 
5 hereinbelow, each input variable can be input to the model with various time delays, such 
that each input variable can in effect constitute multiple input variables to a system 
model. In this manner, the time delay of a variable can be utilized to determine the 
sensitivity of the output on chat input variable at the appropriate time delay. 

The average sensitivity is the average change of the output variable as the input 
10 variable increases from its minimum to its maximum value. A positive Averaee 
Sensitivity value indicates that, on average, the output value increases as the input 
variable increases. A negative Average Sensitivity value indicates that, on average, the 
output value decreases as the input value increases. Average Absolute Sensitivity is the 
average of the magnitude (absolute value) of the change of the output variable as the 
15 input variable increases from its minimum to its maximum value. (Note that this is not 
necessarily the magnitude of the average.) Thus, Average Absolute Sensitivity is always 
positive, and is greater than or equal to the magnitude of Average Sensitivity. Averaee 
Absolute Sensitivity, therefore, gives a general indication of the strength of the influence 
of an input on an output. Combined with Average Sensitivity, the Average Absolute 
20 Sensitivity can be utilized as an indication of whether the input/output relationship is 
linear, monotonic, or without a causal connection. 



Referring now to FIGURES 7a ; 7b, 7c and 7d, there are illustrated plots of 
various illustrations of examples of X-Y plots of an input and an output variable over the 
extent of their range, with the corresponding sensitivity relationships indicated. In 

25 FIGURE 7a, the output decreases in a linear, monotonic manner with respect to the 
input. In FIGURE 7b, the output increases in a non-linear, monotonic manner as a 
function of the input. In both FIGURES 7a and 7b, the Average Absolute Sensitivity is 
equal to the absolute value of the Average Sensitivity, which value is not equal to zero. 
In FIGURE 7c, it can be seen that the output varies in a non-linear, non-monotonic 

30 manner as a function of the input. The Average Absolute Sensitivity is therefore greater 
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20 



than the absolute value of the Average Sensitivity In FIGURE Id, the output does not 
change as a function of the input and, therefore, exhibits no causal relationship to the 
input. In thts manner, the Average Absolute Sensitivity is equal to the Average 
Sensitivity which are both equal to zero. 

In FIGURES 7a and 7b, both relationships are monotonic; that is, the output 
variable does not change direction as the input variable increases. However, the 
relationship does not necessarily have to be linear, since strong non-lineanties can exist 
over small ranges of the input. The degree of non-linearity can be dete.mmed utilizing 
the sensitivity versus percentage analysis descnbed herein. If the relationship is a non- 
monotonic relationship, and therefore non-linear, the greater the inequality, the greater 
the non-linearity. 

Referring further to the calculations of Table 1, it is noted that they are averaged 
over distentions, but if the input/output relationships are quite non-linear, then this L 
hneanty is often not fully delayed in the average or peak relat.onshtp computed in 
Table 1; thus, it :, very useful to compute the sensitivity as a function of the range of the 
mput data. For this computat.on, the sensitivity is computed for several different input 
values in the range of each mput variable. For each input variable, several inputs are 
created as a function of the range of the mput, ranging from 0% to 100%. Thts step is 
performed holding the values of the other vanables constant. For each variable V 
twenty different ranges are computed and the partials at each of the k mput values would 
be as follows: 



Bx k ° W '^^7° W k ; Wya - * nm = Constant 



(10) 



where N, is the number of input vanables and the other x; are held constant at then 
mean peak values for j* k 

The Sensitivity vs. Rank data collapses the sensitivity information into a sinele 
2. number for each input variable (Average Sensitivity), whereas the Sensitivny vs 

Percentage of Range data illustrates the details of how sensitivity vanes across the ranae 
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of each input variable. Unlike the Sensitivity vs. Rank data, this detail allows the user to 
distinguish whether a monotonic variable is linear or non-linear, the extent of the non- 
linearities and at what input values the sensitivity is at its maximum or minimum. 
Sensitivity vs. Percent calculations are only made for one output variable at a time. Once 
5 calculations are complete, the information can be displayed. A sample of this display is 
illustrated in FIGURE 8, which comprises a plot of Sensitivity vs. Percent. This 
illustrates three plots for three separate output variables. All plots are calculated for one 
output variable that was selected in a previous operation. Each line considers an input 
variable over the extent of its range in the Dataset (over all patterns). The X-axis shows 
10 the percentage of the input variable's range: 0% is the variable's minimum value in the 
Dataset; 100% is the variable's maximum in the Dataset. 

The points in the plot at FIGURES 8 and 8a are plotted at intervals of 5% of each 
input variable's range. On each line, that input is held at its particular percentage of its 
range, whereas all of the other model variables are held at their average. The output 
15 (Fig. 8) or the partial derivative (Fig. 8a) of the output versus the input at that point can 
then be calculated. Values on the Y-axis show the output variable's sensitivity to the 
input variable when the input variable is at each different step within its range. 

Referring now to FIGURE 9, there is illustrated a flowchart depicting the 
sensitivity operation, which is initiated at a function block 1 14. The program then flows 

20 to a function block 1 16 that selects rank or percent. If rank is selected, the program 
flows along a branch to a function block 1 18 to perform the Sensitivity versus Rank 
operation. Then, the program flows to a function block 120 to run the model and 
calculate the Sensitivities. The program then flows to a function block 122 to select the 
type of display, Average Absolute, Average or Peak. If the Average Absolute is 

25 selected, the program flows to a function block 124. If the Average Sensitivity is to be 
displayed, the program flows along a branch to a function block 126 and, if the Peak 
Sensitivity is to be displayed, the program flows along a branch to a function block 128. 
After the display has been selected, the output of all the blocks 124-128 flow to the input 
of a decision block 130 to determine if the Sensitivities have been displayed. If not, the 

30 program flows along an "N" path back to the input of block 122 and, if so, the program 
flows along a "Y" path to an Exit block 132. 
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If percent has been selected in ,he block , , 6, ,he program flows along a branch 
.o a Sanction block 134 ,o miflate ,b e program fo, ,he Se„s,,ivi,y versus Percent 
operat.on. The program ,hen flows ,o a funcon block 136 ,„ selec, the desired outpm 
ta. among me outputs associated wi,h ,he output vector, whrch output is the predictive 
output „-(,) The program then „ ou , s , o a fc na . on b|Qck [o ^ ^ ^ 

then ro a nancicn block ,40 to calcubne ,he Sensitivity. The program ,be„ flows ,o a 
dectston block ,42 ,o determine if all ,he inputs have been processed ,f no, ,he 
program flows back to the i„p„, of fiancton block 138 alone an "M" path After 
cotnpletton, the program flows ftom decisiot. block ,42 along the "V path ,„ !he mpu, 
of fianctto, block ,44 to select the mpms to disp„y and then t0 the mpu, of , deciston 
block ,46 to deterge if fc operatton is compleled . Once competed, the program 
flows ftom dectston block .46 along the "V p„h to the input of the Exit block ,32 

In addition ,o determining how sensitive the variables are, the analyzer 30 also 
has the ability to derenrune how these vanab,e, wi„ affec, the output of the plan, ,0 as 
12 .he,, values change. This is facil„a,ed by the "Selpomts" and "What ffi- too , wherein 
,he outputs are de,e™„ed as a funcon „r,h e use , supplled and mod , fied ia 
cetratn constratnts Once a trained system model is loaded, the use, can then change 
mpu, va,ues to predict outputs as , funcon of ch»g,„g inputs. Thus can be dodder 
dtfteren, constraint cond.tions. ,„ ,hi s mode, the oser can constrain valt.es to no, ,a„ s e 
outstde given conditions, clamp mputs to cerram va,ues, and compote the predicted 
outpu, va,„e a,o„g wtth an estimate of „s prediction error fo, a given se, of inputs This 
error ftanctton is refetred to as the "confidence va,ue" , whtch is no, the subject of the 
present a P plioa,ion, bo, was disclosed ,„ U.S. Patent Apphcation Serial Number 980 664 
filed November 24, and ent , tled - Melhod ^ _ ^ ■ 

25 Network with Missing or Incomplete Data". 



The mputs are assumed to be ^dependent variables in a standard model and can 
be set and constrained accordmg to the user's discretion. In this case, the output of a 
model, oP(x) is a function, f(*>, that is learned by the neural network mode. The mput 
values of *«) are gl ven by the user, as well as the constraints on « t) . The constraints 
are of three types: constraints on the absolute range of the values, constraints on the 
tone rate-of-change of the values, and combmatona. constraints on the variables 
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However, other constraints could be implemented. The absolute constraints are 
implemented simply as: 



(11) 



x \x>x . 
' W", 



where, x lowcr and are the lower and upper hard-constraints on the ith input value x,, 
respectively. This relation says to replace Xj with its lower or upper constraint value if it 
5 ranges outside the constraint. 

Similarly, for time-dependent values, the time-rate-of-change on the values can 
be viewed as the difference, or "delta" of the values, A(x) = x(t) - x(t - 1), where x(t) is 
the value of x sampled at time t and x(t -I) is the value of x sampled at one time-step in 
the past, where all of the data is on a constant, pre-determined time interval. For these 
1 0 delta values, the delta constraints can uc implemented as follows: 



*(0 = ] ^(O (12) 

C - ■> ♦ A^;iT(0 - *(i - 1) > Af 



•{ 



This equation says that the time-rate-of-change, or delta- values of the inputs can never 
exceed the delta constraints. 



Combinatorial constraints are used to satisfy limits on the combinations of 
variables. These constraints are "hard" in the sense that they reflect physical constraints 

1 5 in the operation of the plant. For example, just as it is required that the sum of the 

currents into a node of a circuit is equal to the sum of the currents out of the node, so it 
is in a plant. The sum of the flows into a section of the plant must be equal to the sum of 
flows out of that same section. This type of constraint is often referred to as "mass 
balance" or "material balance" requirements, and can be expressed as F^x) = Q, meaning 

20 that some function F;(x) of the inputs x is equal to a constant, C, Other common 
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requirements are "heat balance", meaning the energy in equals the energy out, etc. These 
also can be expressed as F ; (x) = Q. 

To implement combinatorial constraints as a filter on the inputs, it can implement 
it in a manner very similar to the range constraints, pulling the variables back into their 
constrained values any time they stray outside of the specified constraint. For example, 
solve the equation: 

F, *r • X J - C (13) 

for a particular x k) i.e.: 

x t = G. (C , x v x 2 ,... X J (H) 

and set all of the x, for i * k to the values output by the inverse model and then set: 

(15) 



x t = G i (P p x v x 2 . . . x t ) 



This will satisfy the constraint F f (x) = Q and provide most of the values (except for the 
1 0 kth one) so as to minimize the cost. 



Referring now to FIGURE 10, there is illustrated a flowchart depicting the 
operat.on of initiating the Setpoints and What-Ifs operation, which is initiated at a start 
block 150 and then proceeds to a decision block 152 to determine if the Dataset is 
loaded. If not, the program flows along an "N" path to a function block 154 to load the 
Dataset and then to the input of a decision block 156. If the Dataset is already loaded, 
the program flows directly to the decision block 156. The decision block 156 determines 
whether the model is loaded. If not, the program flows to a function block 158 to load 
the model and then to the input of a decision block 160. If the model is already loaded, 
the program flows directly to the decision block 160. Decision block 160 determines 
whether the view needs to be changed. If yes, the program flows to a function block 162 
to change the view and then the input of a decision block 164. If the view does not need 
to be changed, the program flows directly to the input of a decision block 164. The 
decision block 164 determines which mode is selected, the Predict Inputs mode or the 
Predict Outputs Mode. If it is the Predict Inputs mode is chosen, the program flows to a 
function block 166 to predict the inputs and then to a decision block 168 to enter a loop 
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until the inputs are predicted. After prediction of the inputs, the program flows along a 
" Y" path to the input of a decision block 1 70 to determine if more analysis is required. If 
so, the program flows along a "Y" path back to the input of decision block 160. 

If the Predict Outputs path is chosen from decision block 164, the program flows 
5 to a decision block 172 to perform the output prediction. The program then flows to a 
decision block 174 to enter a Do loop until the output prediction is complete, after which 
it flows from decision block 174 along the "Y" path to the input of decision block 170. 

After the analysis is complete, the program flows from decision block 1 70 along 
the M N" path to the input of a decision block 1 76 to determine if the model parameters 
10 are to be saved. If so, the program flows to a function block 178 to save the model and 
then to a Done block 180. However, if it is not to be saved, it flows directly to the Done 
block 180. 

Referring now to FIGURE 1 1, there is illustrated a flowchart for the operation 
wherein the outputs are predicted, which is initiated at a block 206 and then flows to a 
15 decision block 208 to determine which of three branches is selected to determine the 

source. These can be either the Dataset, the averages of the dataset, or the screen. If the 
Dataset branch is chosen, the program flows to a decision block 210 to determine if new 
parameters are required. If not, the program flows to the input of a function block 212 
and, if so, the program flows to a function block 214 to set the input parameters and then 

20 to the input of function block 212. Function block 212 is the operation wherein the 

values are retrieved from the Dataset which, after retrieval, the program will then flow to 
a decision block 216 to determine whether the program is to be run. If not, the program 
will flow to the input of a decision block 218. However, if the outputs are to be 
predicted using the Dataset, the program flows to a function block 220 to run the model 

25 in a forward pass and then to a decision block 222 to determine if the forward pass is to 
be stopped. If not, more values are fetched from the database and then the program 
proceeds back to the input of function block 220. However, after the model has been 
run in the forward pass, with all the data, the program flows from the decision block 222 
to the input of decision block 218. Decision block 218 determines whether the step 

30 function has been selected from the graphics buttons 196. If so, the program flows to a 
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function block 224 to run the model in a forward pass and then to a function block 226 
to get additional values from the Dataset and then back to the input of decision block 
21 8 to wait for the next step. Once this is completed, the program flows to a return 
block 228. 

If the Averages path has been selected from decision block 108, the program 
flows to a decision block 230 to determine if new parameters are required If so, these 
parameters are set, as indicated by a function block 214, and then the program flows to a 
function block 234. If no constraints are required, the program flows directly to the 
function block 234, which function block 234 then fetches the value from the dataset 
averages, which were determined in the training operation. The program then flows to 
the input of a decision block 236 to determine if the Step button in the graphics buttons 
group 196 has been selected. If not, the program flows to the input of the return block 
228 and, if so, the program flows to a function block 238 to run the model in the forward 
pass, i.e., to perform a prediction, and then to the return block 228. 

1 5 If the Screen source had been selected at decision block 2CS, tne program would 

flow to the input of a decision block 240 to determine if new values are required. If yes, 
the program flows to a function block 242 to receive the values from the screen, these 
input by the user. The program would continue in a loop back to the input of decision 
block 240 until all values were entered and then would flow to a decision block 244 to 
determine if new parameters are required. If so, the program flows to a function block 
214 to set the constraints and then to the input of function block 248. If no new 
parameters are required, the program flows directly to function block 248, wherein the 
values input by the screen are then retrieved and the program flows to the input of 
decision block 236 to run the model. 



20 



25 



30 



Referring now to FIGURE 12, there is illustrated a flowchart depicting the 
operation where.n the input parameters are changed, which is initiated at a function block 
214 and then flows to a function block 252 to select the input variables. The program 
then flows to a decision block 256 to determine if new input parameters are required. If 
so, the program flows to a function block 258 to change the input parameters and then 
back to the input of block 256 If no constraints are required, the program flows to the 
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input of a decision block 260 to determine if a new delta constraint is required. If so, the 
program flows to a function block 262 to change the delta constraints (rate-of-change 
constraints) and then back to the input of block 260. If no new delta constraints are 
required, the program flows to the input of a decision block 264 to determine if the 
5 inputs should be clamped. This operation indicates that the input values should not 
change. If so, the program flows to a function block 266 to perform the clamping 
operation and back to decision block 264. If the inputs are not to be clamped, the 
program flows to the input of a decision block 268 to determine if the Scroll Plot should 
be illustrated. If so, the program flows to a function block 270 to turn on this feature 

1 0 and then back to the input of function block 268. If the Scroll Plot is not to be displayed, 
the program flows to the input of a decision block 272 to determine if the Confidence 
Interval is to be changed. As described above, this is associated with the error of the 
predicted value, which is the subject of another and pending application. However, if the 
Confidence Interval is to be changed, the program flows to a function block 274 to 

15 change the Confidence Interval and then back to block 274. If the Confidence Interval is 
not to be changed, the program flows to a return block 276. 

Additionally, rate-of-change constraints (delta constraints) can be provided by 
inputting two values, a max-decrement and max-increment. The decrements are set in 
accordance with the following equations: 



cu 



£ X 
. * UCJ 

< x. (r + 1) 



* (0 * A 



CUl 



(16) 



20 where: A CLI = Max - Decrement 

A cm = Max - Increment 

As can be seen from the above equations for the rate of change constraints, the amount 
that the input variable can change for each "Step" through the network will be set. This 
provides for an additional level of control over the predicted control variables. For 
25 example, if it is desirable to change a flow setting, the control input device, such as a 



WO 94/25933 



PCT/US94/04€92 



28 



valve, is not immediately changed to a new setting but, rather, it is incrementally 
changed. 



Refernng now to FIGURE 13, there is illustrated a flowchart for the operation 
where,n the inputs are predicted, which is initiated at a function b.ock 332 and then flows 
> to a decision b.ock 334 to deterge one of three branches from winch to select the 
source, the Dataset, the averages or the screen. If the Dataset is chosen, the program 
flows to a decision block 336 to determine if new parameters are to be applied and if so 
the program flows to a function b.ock 2 14 to set the constraints and then to a decision ' 
block 340. If not, the program flows directly to the decision block 340, decision block 
10 340 determining whether the output parameters are to be set. If so, the proeram flows 
to a function block 342 to set the output parameters and then to a function block 344 If 
the output parameters are not to be set, the program flows directly to the function b.ock 
344, the funcdon block 344 denoting the operation wherein the values are obtained from 
the Dataset in the memory 26. The program then flows to a decision block 346 to 
1 5 deterrrnne whether tbe run operation has been activated. If so, the program flows along 
a Y path to a function block 348 to run the inverse mode, (run mode, in a backward 
pass) and then to a function b.ock 350 to determine if the step operation has been 
selected. If not, the program flows to a function b.ock 352 to get additiona. values from 
the Dataset and then back to the input of the function b.ock 348 to continue the Run 
20 operation Tms wi.. continue unti. a Stop function has been indicated, after which the 
program wi,. flow to a decision b.ock 354 to determine if the Step operation has been 
selected. If so, the program wi.l flow to a function b.ock 356 to run the model in a 
backward pass and then to a function block 358 to obtain additional va.ues and back to 
the ,n P ut of the decision b.ock 354 to wait for the next step. If another Step is not 
25 selected and the program is ha.ted, the program flows to a Return b.ock 360. 

If the source is the averages database, which is determined in the sensitivity 
operation, the program will flow to a decision block 362 to determine if new parameters 
are to be selected. If so, the program flows to a function block 214 to set the input 
parameters, and then to a decision b.ock 366. If not, the program flows d.rect.y to the 
30 decision b.ock 366, winch decision b.ock 366 determines if the output parameters are to 
be set. If so, the program flows to a function b.ock 342 to set the output parameters and 
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then to a function block 370 to get the values from the area in storage wherein the values 
are stored after processing through the training operation. If the output parameters are 
not to be selected, the program will flow directly to the function block 370. After the 
values have been retrieved, the nrogram flows to the input of a decision block 372 to 
5 determine if the Step function has been selected. If not, the program flows to the Return 
block 360. However, if the Step operation is to be utilized, the program will flow to a 
function block 374 to run the model in a backward pass and then to the Return block 
360. 

If the source is the screen, the program flows from decision block 334 along the 
10 "SCREEN" path to a decision block 373 to determine if new values are required. If yes, 
the program flows through a loop including a function block 375 to enter values from the 
screen and then back to the input of decision block 373. If no new values are required, 
the program flows to a decision block 377 to determine if new parameters are required. 
If not, the program flows to the input of a decision block 379 and, if so, the program 
1 5 flows through a function block 214 to set the input parameters and then to the input of 
the decision block 379. The decision block 379 determines whether output parameters 
are to be entered. The program then flows to a function block 383 to get the value from 
the screen. If output parameters are to be input, the program flows to a function block 

379 to set the output parameters and then to the function block 383. After the values are 
20 obtained from the screen, the program flows to the decision block 372. 

Referring now to FIGURE 14, there is illustrated a flowchart depicting the 
operation wherein the output parameters are changed, which is initiated at the block 342 
and then flows to a block 378 to select the output variable and then to a decision block 

380 to determine whether the fuzzy-constraints are required. If so, the program flows to 
25 a function block 382 to change the fuzzy-constraints and then back to the input of 

decision block 380 until all fuzzy-constraints are changed and then the program flows to 
a decision block 384, where it is determined whether new cost coefficients are required. 
If so, the program flows to a function block 386 to change the cost coefficients and then 
back to the decision block 384 until all cost coefficients have been changed. The 
30 program then flows to a decision block 388 to determine if new desired values are 

required. If so, the program flows to a function block 390 to change the desired values 
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and .hen back ,„ , he i„ put of deasio „ b|ock ^ ^ ^ ^ ^ 

The prog™ will lhe „ „ ow to , deci$ion b|ocfc J92 w deKnmne . f ^ ^ _ 

are reared ,f s „, the program „ ows ,„ , ^ ^ ^ ^ ^ ^ ^.J 
ranges a„ d then back t0 the inpu , of dec , $jon ^ ^ ^ ^ ^ ^ 

The program wil, ,he„ flow through . decision block J06 , 0 jf ^ 

Pious ,o be i„u s , ra ,ed The Scroli P,o, is a plo, wherein ,he pred,c,ed on.pnr is 

agains. ,he »u>, targe , value or the ^ values ^ ^ 
on , e vanoos oo.pu, papers If s0 , the program „ ows to , ^ 
,sp a y the Scrofl P,„, and ,hen hack ,o ,he i„ pul of deasion block 39 , ^ „ ^ ^ 
■spiayed, t be progran, flows t o a dec,s,o„ block 400 ,o derempne if ,he Confidence 
« ou ,d he changed. ,f so, the progfam flows , 0 , ^ b|ock ^ ^ 
= Confidence taerval and ,hen hack ,o «he decision block 400 ami, ,, has been " 
changed, after which ,he program flows to a return block 104 

s p'ofT' as des ? ed above ' de,erames how ,he — g « » 

operare Often, ,„ a nvanofacronng process, „ is useful ,o de.eno.ne ,he appropria.e 
■npurs ,o aehaeve a desired ourpu, cnKno , Thjs „ „ be , 

i:: 1 ,^"- B -*™-^^.cft.eHo n ,o„eca„de 1 er mjne 
mcrease ,he y ,e,d of a cenarn prodocon. decrease ,he e„e rgy consu mpt ,o n , decreasc fc 

•he user rnus, provide i„ foimatlon „ t0 wha , ^ ^ ^ ^ * ■ 

-de, of.be plan, ,s fl™ Werted" t0 predlct fc ^ ^ ^ ^ ^ 
OU.PO, <„ a sense, .bis provrdes a de.ernnnanon of ,he proper -recipe- for .he process 
For a s.ead,s„e so,u,ion, .his is referred ,o as process oprinnaanon ,f lhis 
condnron . isfied , n an o„-i,„e m . th en M , deIerrana ,, 0 „ ^ .^J^ 
referred to as Process Control 



." genera,, fine desired behavior can be an arbnrarflv complicated ^ 
,pu,s and „ U , PUIS Xha , „ one Qn wn[e an op(im% toctim w a coa _ ta 

.he nnpurs of ,he p,a„, M0 and lhe p _ q This „ ^ , q ^ 

cost function, which can generally be stated as follows; 
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Npats 

E co« - £ f (m, *, c) (I7) 
k = i 

Where E cc „ is the total cost, F( ) is the cost function and N pals is the number of patterns 
the total cost is evaluated over. Since the actual outputs of the plants are not available 
during the analysis run, they must be predicted with the model, i.e., it is assumed that the 
model realized with the neural network is an accurate model, y(x) = o(x). With this 
predicted output value, the cost function can be written as follows: 

Npats 

E cost = E F (0 (X), X, C) (18) 

k = 1 



In the preferred embodiment , an explicit cost function can be written to satisfy 
different criterion. However, it should be understood that multiple criteria could be 
utilized to modify or modulate the process. In the preferred embodiment, the cost 
function is divided into different pieces for each criteria as follows: 

£ cost.t = E i, k + E 2k + ... E lk ^ 19 j 

where, each of the terms is the mth error function term for the mth cost criterion 
associated with the mth property of the plant, for the kth pattern. 

Explicitly, to achieve the desired value d } on an output or predicted state variable 
o„ the first cost criterion can be written as the following: 



Nout 
i = 1 



1. 1 



Nout 
i = 1 



(20) 



Where the sum is over the number of outputs N oul . This term has a cost-coefficient A ; so 
1 5 that each output can be weighted differently for different desired properties, and a 
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•fczy-cos-coeffiden,-. { („,), described below ,„„, „ lows [he ^ ^ ^ 
ranges This fi.ncon E, wil, be m,„, mum ifall of Ihe outp „, ^ ^ ^ ^ 
v^ue, Note ,ha t addinonal desired va,ues ean be associa.ed wi,h ,he i„p u , vanaWe as 
well. In this case, another term would exist as follows: 



Nin 



i = I 



/V/>7 

i = 1 



(21) 



10 



15 



where, ,„ t h,s case. ,he sun, is over «he ,„p ut pa„er„s Again, ftas cost would be 
manned if ai, ,he valnes are e, ua , ,„ fte desired values If Ms „ no , possltfc as „ 
often the case ,„ real-world si.uanons wift confticing desires, an arremp, is ma de to 
manage ,he cos, function such ,ha, t he ,o,a, su m -s„ua,e.error is minted over all 
pa„.ms. It is also the case t ha, the user does no, have s,nc, cxac, values for ,he destred 
propeces of ,he process, ye, a desired range of values exist To achaeve ,he beh.vror .„ 
a grange", ,h« fuaay-cos.-coeftaen, W can he used Thus „ facI , ons 10ta|Iy 
analogous,, ,o fte overall e^-coefficen, A„ b«, for ,he foaay coefficn,, ,he cos, 
vanes as a ft,„c,,o„ of ft. ou, pu , para m e,er, decrea S ,„g ,„ aero a, ,„e exact desued 
value, and .ncreasing ,o one a, ,he edges of fte destred ranges Tha, is, fte fiaaav-cos, 
coefficen, is calculated as firncion of range as 



/, <=,> ■ 



1 " /««f«f 



I ~Jow 



< <d t ) 



' ' uppir 1 



(22) 



' VP", 



This provides a fuzzy-range value with the 
from the des.red-vaiue, where z,can be any one of the 



cost increasing as the value gets further away 



inputs or the outputs, or predicted 
state variables. 
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The system can also implement "soft constraints" or "frizzy-constraints" that 
allow the user to specify a range of values which he desires the process to avoid. The 
cost terms of choice for this are as follows: 

Nout Nout 

E > - z e u - z «, (-•, P) - <.,„,,) e (.-, (x) - .- wr() (23) 

/ = 1 I = 1 x 7 



3 



Where, c uppcr ; is the upper fuzzy-constraint limit on the variable, and 0(z i ) is the 
Heavyside Step Function, G(Zj) = 1 for z,. non-negative, Q(z i ) = 0 otherwise. Similarly, 
there is fourth term to implement the lower constraints: 



Nout Nout 

E * t_ ^ E <j Z ^ Pi <<w, - =< (*)) © (C w , - r, (*)) ^4) 



Where, c, owcr <s is the lower fuzzy-constraint value and p is a cost-coefficient. Thus, the 
total cost function is written in the present embodiment as: 



(25) 



This cost function can be minimized via any one of the standard function-minimization 
10 techniques such as gradient-descent, simulated annealing, random-search, etc. In the 
preferred embodiment, the gradient-descent is utilized. In this process, the gradients of 
this cost function are calculated and the inputs x(i) are changed incrementally so as to 
minimize the total cost. This set of values will then be the new "recipe" to optimize the 
plant operation to achieve a desired set of properties. 

1 5 With further reference to FIGURE 3, it can be seen that the hard constraints 

placed on the input values are associated with the filter 46. The filter 46 allows the user 
to download parameters that will prevent the input parameters from exceeding certain 
values, and will also place constraints such as rate of change on the input variables. As 
described, these are "hard" constraints, which should be differentiated from "soft 

20 constraints" or "fuzzy-constraints". The soft constraints are constraints that can be 
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violated, whereas the hard consents are constrant.s tha, canno, be viotaed For 
example, ,he syst e m should never provide a se,poi„, with , value lha( , s physi 
..possible. Therefore, .his would he a hard c„„s,ra, ra However, a plam operaIor may 
deter™ ,ha, the operabon of ,he valve above o, below certain ,«vels is oueshonable or 
tnurebable and i, would rherefore he des„ab,e to naairdarn .be va.ve settings wrrhrn certa,„ 
soft consrraints" AddMonaUv, .here m a y be cena.n co„s„ai„,s .ha, need ■„ be placed 
on .he ourpu, of ,he plan, ,0 These can be .ncorporated ,o the con.ro, „e, operaftou by 
P.ac,ug ,hesc co„s,ra,„,s o„,o ,he predicted ou.pu, These soft co„s,rai„,s on ,he inpu, 
vanab as and ,b= pred.c.ed ou.pu, varrables are effected by ,he cos, nu^aauon bloch 
42, whach perforuts ,he var.ous calculauons described above These co„s,ra,„,s are 
e,™„ed bv .be user dunng Ihe analysis pnMdure a „ d ^ ^ ^ 

i and 53 associated wi,h , he cos, coefficients, fuazv constraints and desired values ' 
respectively. 

As an example of a situation where,„ cos, coefficients wouid be uuiiaed, suppose 

■ > ^ .he sy s,e m co„ sl s,ed ofa process ,ha, operated to outpu, such ,bd„ gs as tapuri 

■ co_„s wfth ,„p„, vanables such as .enrperature, flow rates and p„ mp spMs 
Add,„o„a„ y , i, is tawn that £emin of , he . nput variab|ej ^ ^ ^ 

se,.,ug, have hagher costs associated tnerewnh. For example, i, mght be kmvm , ba , 
vanyrng the p„ mp speed would be extre m e. y expensive due to the overall iogistics 

in ,here ™ ,h ' as compared ,o v ^ ing a now ~ ^ «** • ~ 

nra y a,so be expense due ,„ ,be efficie»c y rvrth whjch a hea.er con.rol could dehver a 
cena,„ antonn, of beat ,o ,h. process As such, these could be idenr.ned and venous 
^.coefficients associated .nerewrth For example, ifit were „ ow „ ^ , 

had no beanng on overall costs, then tbJs could have a low bngh cos, coefficient of 
for exannpie 0,05 On the other band. v aryi „ g the p ump speed may be . ve[y ^ ' 
■nefficen, operation and this could have a relatively high cos, coefficient associated 
^erewbh of, fer ^ 0 ,5. „ .ay als0 be _ lhat fc flow _ fe 
w than cena,„ reg.ons, and above or be|ow ^ ^ ^ ^ 
T e would be , he ^ consirajms ^ ^ sa m juch ^ hQugh ^ 

he above e q ua,,on S a s being step nancnons, they could be any tvpe of linear or no, 
linear function. 
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With respect to the output variables, it might be known that the impurity 
concentration of a given product, which impurity concentration is the measured value of 
the product, has a cost factor associated therewith. For a high impurity concentration, 
the impact of the change may be relatively high, whereas, for a low impurity 
5 concentration, the impact may be the opposite. By way of example, in semiconductor 
processing, it is always, of course, desirable to have a 100% yield. However, this is not 
practical and a more realistic yield percentage would exist. Since real world constraints 
require the yield to be above a defined minimum, which is a "break even" yield. 
Therefore, although it is desirable to have a 100% yield, a yield of 45%, for example. 
1 0 may be acceptable, especially if it relieves some of the constraints on the other 

processing parameters of process. It would not, for example, be cost efficient to increase 
the yield from 45% to 75% if this required an increase in the use of an expensive initial 
processing gas of, for example, Argon. These are relatively expensive gases and it would 
be desirable to minimize the use of this gas, even though the percent yield would go 
1 5 down, since the overall cost would be improved. Therefore, it can be seen that the 
desired values of the output and the inputs are not hard constraints. 



Referring now to FIGURE 15, there is illustrated the plot of two output variables 
o,(t) and o 2 (t) and a plot of two input variables x,(t) and x 2 (t). The cost coefficient A, 
for the first output variable o,(t) is set equal to 1 .0, whereas the cost coefficient A, for 
20 the second output variable is set equal to 0.4. There are two plots, a solid line and a 

dotted line, the solid line representing the desired output for the plant and the dotted line 
representing the predicted output with the cost coefficient applied. It can be seen that 
for the case where A is set equal to 1.0, the predicted output and desired output were 
essentially the same. However, for the case where k is set equal to 0.4, the predicted 
output deviates from the desired output or the error therebetween is considerably higher 
with respect to the output variable o,(t) This is the same situation with the input 
variables x,(t) and x 2 (t), where the cost coefficients are set equal to 0.6 and 0.8, 
respectively. It can therefore be seen that by providing some type of cost constraint on 
the input variables and the output variable during the prediction operation that predicts 
30 the control inputs, the actual predicted control inputs may vary from an actual desired 
input or output. For example, in a conventional control net, a desired output is input to 
the net and the input variables forced to values that will result in the predicted output 



25 



WO 94/25933 



PCT/US94/04692 



36 



equalling the desired output. However, this may not be the most cost effective way to 
run the process. Therefore, it can be seen that the control net of the present invent.on 
will pred,ct input vanabies in accordance with user denned cnteria that will yield 
predicted input variables that will operate the plant under this criterion. 

Referring now to FIGURE 16, there is illustrated a plot of the fuzzy- constraint 
fi.nct.on f(o,) as a function of the value of o. It can be seen that the value of this 
function is minimum at the desired value d, and increases in value as the value of o 
deviates from d, At the lower constraint, f LC , and at the upper constraint f uc , the value 
maintains a constant value 



10 



15 



20 



Refernng now FIGURE 1 7, there is illustrated a block diaeram of a neural 
network 406 that is trained on selected inputs, which inputs are selected as a function of 
the sensitivity of the output on the entire Dataset of the input variables. The neural 
network 406 is a conventiona. neural network composed of an input layer 408, an output 
layer 4,0, and the hidden layer 4,2. The input layer 408 is interconnected to the nodes 
and the h,dden layer 412 through an interconnection network with the outpu, layer 410 
similarly interconnected to the hidden layer 4 1 2. The hidden layer 4 , 2 is operable to 
map the input layer 408 to the output layer 4,0 through a stored representation of the 
Plant 10. As wi„ be described in more detail hereinbelow, the neural network 406 is 
configurable since the number of hidden nodes, the number of input nodes and the 
number of output nodes can be varied, with the associated interconnects also being 
vaned. This is a conventiona. Backpropagation network. However, any type of neural 
network that provides a non-linear representation of plant 10 could be utilized. 

The neural network 406 is controlled by a network configuration/tram system ,4 
which ,s operable to configure neural network 406 as to the number of hidden nodes the 
we,ghts, etc. Of course, the weights are .earned dunng a training operation wherein an 
error ,s developed between a target output and a pred.cted output and this error 
Backpropagated through the network to set the various weights. The network 
configuration/train system ,4 is controlled by a processor 416. The processor 4,6 is 
operable to interface with a database 4,8 that contains a piurality of training patterns 
30 whrch are referred to as the Dataset The Dataset 4.8 is connected to the processor 
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through a line 420, such that both input data and target data can be processed, the input 
data and target data operating in pairs. 

During the training operation, the input data is selected by an input select block 
422, which is interfaced through the database 418 through a line 424. The input select 
5 device 422 is operable in two modes, a first training mode wherein all the input data in 
the Dataset is selected, and a second mode wherein only select data is input to the neural 
network 406. The select data is determined during the sensitivity operation, wherein 
only the input data having the most effect on the output is selected, as will be described 
hereinbelow. The output of the input select circuit 422 is input to the input layer 408 as 

10 an input during the training operation. A multiplexer 426 is connected to the output 
layer 410 and is operable to input target data to the output layer 410 during the training 
mode, which target data is received from the processor 416, this target data retrieved 
from the database 418 by the processor 416. Additionally, multiplexer 426 is operable to 
output the predicted output o p (t). This output is input to the processor 416. The 

15 processor 416 is operable to control the multiplexer 426 to operate in the training mode 
or in the prediction mode, and also to control the input select device 422 to operate in 
the first or second mode associated therewith. 

During learning, the neuronal units in the hidden layer, the hidden units, 
are mapped onto the input space in the input layer in a non-linear mapping procedure. 

20 However, in conventional neural networks utilizing multiple layers with at least one 
hidden layer, all possible layered environments are produced, thus requiring a larse 
number of hidden units to effectively cover the input space. In order to adjust the 
weights and activation parameters of the neural network, a learning algorithm must be 
applied. One of the more widely used learning algorithms is the Back Propagation 

25 method, which is described in U.S. Patent No. 4,893,255, issued to M. S. Tomlinson, Jr. 
on January 9, 1990, which is incorporated herein by reference. This is also described in 
D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning Internal Representations by 
Error Propagation" (in D. E. Rumelhart & J. L. McClelland, Parallel Distributed 
Processing, Vol. 1, 1986), which is incorporated herein by reference Back Propagation 

30 is essentially the backward propagation of error through the network with the changes 
being proportional to the error signal at the output of the network. Essentially, the error 
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» firs, calculated fot the output layer and thcn thjs mm ^ ^ ^ 

wetgh, changes for „„i, s thal feed ,„,„ lhe ^ ^ ^ ^ ^ ^ ^ 

success™ layers until the process terminates back a, the i„p„, layer 

In operation, the neural network 406 is initialiy configured to accept all innu, 
' vanables ■„ the input layer 408 sueh that there is node for each input Therefore a 

r=qu,s„e number of htdden nodes win be reared in order to ,„,ercon„ec, with the 
mnxunun, „ umber of inpuI „ odes in input , ayer 40g m ^ ^ ^ 

changed, the output typica „ y will rentain the sanre for any co^ mm of Ihc ne|woft 
406, Dunng the firs, .node, ,he neural network 406 is trained on a„ of the data patterns 

w,,h an i„ P „, variables inpm theret0 However cmm vamb|ej ^ ^ ^ 

■fie output, i, e „ se „s,, M , y of lhe oulpuI „ , glve „ ^ ^ my ^ 
zero or of insignificant effect. 

Afler the neural network 406 is trained on all ,„pu, vanables, the input dara is 
•hen processed through the sensitivity algonthuns descnhed hereinabove wi.fi respect to 
Average Sensinvfiy, Absolute Average Sensing, and Peak Sensitivtty, Since the neural 
network was tratned .firougfi a Backprop.gation .echunque, the pan.al denvarives 
already exist and the processor. These are typically stored in a ntetnorv 428 The 
processor 416 need only ^ ^ ^ ^ ^ ^ 

the .rating operation and stored ,„ rhe nr em o, 428 in order to calculate the sensitwaues 

however, tha, the sensirivity calcnlations are no, dependent on the mo dc, being .rained ' 
wt.fi Backprop.ga.ion. The sensi.ivines of any m od=, „<„) can be calculated by 
nunnenca, approrama.ions to .fie denvitives as follows: Additionally, ,he n^ 428 is 



(26) 



operable ,o s,„re user defined .bresho.ds, wfiucfi user defined .fieesholds de.en.ne wfia, 
« se„,.,vi,ics ,rc ,f , he sensitivity of , gjve „ ^ on ^ ^ 

U below a predc.enn.ncd .fireshold, .bis ,„ put vanaWe , s ,„ be ^ 

and no, necessary .o provide an adequate repression of the systent However i, is 
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important to note that the sensitivity operation is performed on a learned representation 
of the output on all of the input of variables, such that the combined effect of all other 
input variables are taken into account in this stored representation. If this were not the 
case, then any direct correlation function could be utilized wherein a single input variable 
5 were varied to determine the effect on the output. This is not acceptable since the 
system is non-linear and it is possible that a given input variable may have very little 
effect upon the output by itself, but when taken in combination with another input 
variable, the effect may be dramatic. The above equations relating to sensitivity take this 
into account. 

1 0 Once the acceptable input variables have been determined (those that have an 

appropriate effect on the output), the input select circuit 422 is operated in a second 
mode and the input layer 408 reconfigured to have only the number of input nodes 
associated with the selected input variables selected by the input select circuit 422. The 
network 406 is then again trained with only these selected input variables. Thereafter, 

1 5 this network can be utilized to provide either a predictive network or a control network 
function, wherein the inputs are predicted. However, it is important to note that the 
operation of this network accepts only the limited number of select inputs. Although the 
total number of inputs received from the system are not incorporated into the updated 
network, it has been observed that this updated network with fewer number of inputs 

20 provides some degree of generalization in the training of the network. 

Referring now to FIGURE 18, there is illustrated a flow chart for the training 
operation for the neural network 406. The program is initiated at an initial Start block 
430 and then proceeds to a function block 432 to select all input variables for input to 
the untrained network. The network 406 is then configured accordingly and trained in 
25 accordance with the Backpropagation training algorithm, as indicated by a block 434. 
The program then flows to block 436 to freeze the weights of the network and then to a 
function block 438 to calculate the sensitivities for each output variable. These 
sensitivities are then stored and the program flows to function block 440 to select the 
desired inputs. 
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Dunng the selection of the desired inputs, the system can either do an automatic 
selection or a manual selection. This is indicated by a decision block 442. If automatic 
selection is chosen, the program will flow to a function block 444 to compare the 
sensitivities of the output on various input variables with thresholds. Any of the 
5 sensitivities that exceed a predetermined threshold will be selected and the others 

deselected. The program then flows to a decision block 446 to determine whether the 
inputs are selectable and, if so, the program flows along a "Y" path to function block a 
448 to select and store the input vanables and then the program flows to a decision block 
450. If the value is not acceptable, the program flows from decision block 446 along the 
1 0 "N" to the input of the decision block 450. 



The decis.on block 450 determines whether the operation is done, i.e, whether all 
inputs have been compared to the thresholds. If not, the program flows along the "N" 
path to the input of a function block 452 and then back to the input of the function block 
444. When ail input vanables have been tested for each of the output vanables relative 
to the sensitivity, the program flows to a function block 454 to train the network only on 
the selected input/output pairs. Alternatively, the deselected input/output pa,rs could 
have weights associated therewith forced to a "0" value during the second training 
operation such that the network would never tram on these input/output pairs. The 
program would then flow to a return block 456. 



If the manual operation is chosen, the program would flow from the decision 
block 442 along the "Y« path to the input of a function block 458 to select desired input 
variables. Thus can be done by a user examtrung vanous display plots of the sensitivity of 
the output variables as a tunct.on of the input vanables and then make user-defined 
judgments about the data. After selectton of the input vanables, the program flows to 
the input of function block 454 and then the neural network 406 will be trained on only 
those selected variables. 



Refernng now to FIGURE 19, there is illustrated a detail of the neural network 
406 with only select inputs and outputs illustrated The select input vanables can be any 
one of the input variables in the Dataset in addition to delayed forms of that vanable 
30 One technique for defining delayed forms of an input variable is described in U. S. Patent 
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Application Serial No. 008,218, filed January 25, 1993, and entitled "A PREDICTIVE 
NETWORK WITH LEARNED PREPROCESSING PARAMETERS" In general, the 
input variable x,(t) is passed through either a delay block 460 with a delay t , or a delay 
block 462 to provide a delay t 2 . The output of delay block 460 will be delayed input 
5 variable x,(t + x x ) and the output of delay block 462 will be the input variable x,(t + t,). 
It will be appreciated that each of these input variables, although related to a single input 
variable by a delay, will elicit a given response in the output variables such that each of 
the input variables, whether they are delayed from a given input variable, will have a 
different sensitivity. The output variables from the output layer 410 can also be delayed. 
10 For example, the output variable 0 2 (t) can constitute one output variable with another 
output variable 0 2 (t + ? 3 ) provided by passing the variable 0 2 (t) through a delay 
lock 464. 

Referring now to FIGURE 20, there is illustrated a block diagram of the control 
function wherein control inputs are predicted and varied through the use of the sensitivity 
15 operation. As described above, with respect to FIGURES 17 and 18, the neural network 
is trained and utilized as a control network 470. This control network 470 is similarly 
described above with respect to FIGURE 3 wherein both a plant model 54 exists and an 
inverse plant model 56 exists. The control network 470 is operable to receive the input 
variables x(t) and the state variables s(t). In addition, the control network 470 must 

20 receive some type of desired value, as indicated by block 472. The desired value, in 
addition to the other inputs, allows an error to be generated, which error is then 
propagated through the inverse model in the control network 472 to generate a predicted 
input x(t + 1). This is output on a line 472. In a typical operation, the updated variable 
x(t + 1) is input directly to DCS 14. However, each of the input variables in x(t + 1) is 

25 modified by the sensitivity of the output variables on that input variable. For example, if 
certain input variables such as pump speed are determined to have little effect on the 
output, a prediction by the control network 470 that this input variable should change is 
modified as compared to the other variables wherein the output has a higher sensitivity to 
those other variables. In this manner, unnecessary changes in the pump speed can be 

30 prevented, since this has little or no effect on the output, even though the control 
network 470 predicted that this should change. Other parameters, such as a civen 
temperature, may have been determined to have an output variable that is very sensitive 
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to that temperature. As such, a predicted output indicating that the control input 
associated with the temperature measurement should be changed will be given maximum 
priority. Additionally, it may be determined that for certain ranges the pump speed has 
little effect on the output, but that for other ranges of input values of the pump speed, a 
greater effect on the output will be present. This could exist where, for example, a 
hjgher pump speed caused the pump to cavitate, thus having little or no effect on the 
output. By not changing the pump speed under these conditions or in this range, a more 
efficient system can be realized. 

In order to modify the input variables, a sensitivity controller 476 is provided that 
controls variable modifier blocks 478 for each of the input variables. The input variable 
x,(t + 1) is input to the associated variable modifier block 478 and processed through a 
fiinction f(x,(t + 1)). Similarly, the remaining variables x 2 (t + 1) through x M (t + 1) also 
modified by appropriate functions f 2 ( t + 1) through f N (t + 1 ). 

In summary, there has been provided an analysis network for examining the 
1 5 operation of a distributed control system in the operating mode. The analysis network 
operates in the background independent of the operating control system and is operable 
to determine what parameters can be changed in order to produce certain desired effects 
or to merely examine the operation of the plant under certain artificial conditions One 
aspect that can be analyzed is the sensitivity of various output variables on different 
values of the input variables. By determining such sensitivities, certain input variables 
can be determined as having no effect on the output and can therefore be eliminated from 
the network. In one mode, the sensitivity measurements can be utilized to modify the 
predicted input values for each of the input variables, such that the most sensitive input 
variables are given priority and passed through at their maximum predicted value, 
25 whereas the less sensitive input variables have the predicted values modified to pass 
through only a percentage of their predicted value or none of their predicted value. 
Alternatively, the actual control network itself can be built utilizing the sensitivity 
measurements. In this mode, the neural network is first trained on all of the input 
variables and the associated test patterns of input/output pairs, and then the sensitivity of 
30 the output variables on the input variables determined. The less sensitive input variables 
are then deselected and the network reconfigured to have only input nodes associated 
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with the selected input variables. The network is then trained only on the selected input 
variables. 



Although the preferred embodiment has been described in det?<l, it should be 
understood that various changes, substitutions and alterations can be made therein 
without departing from the spirit and scope of the invention as defined by the appended 
claims. 
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WHAT IS CLAIMED IS: 
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1 A system for determining a set of sensitive inputs in a non-linear 
predictive network having stored therein a representation of a plant, comprising: 
a configurable network having: 

an input layer for receiving a defined number of input 
variables, 

an output layer for outputting a defined number of 
outputs, each associated with an output variable; and 

a hidden layer for mapping said input layer to said output 
layer through a stored learned representation of the plant; 
a dataset of training patterns representing the input variables to the plant 
and associated measured output variables of the plant, with each of said training patterns 
having input values associated with sa,d input variables and corresponding output values 
associated with said output variables; 

a training system for training said configurable network in accordance 
with a predetermined training algorithm; 

a sensitivity processor for determining as a sensitivity value the effect of 
each of said input vanables to said configurable network on a learned representation of 
said output variables, wh«i„ the combined effects of all other input variables are taken 
20 into account; 

a central processor for configuring said network to receive all of said 
input variables in a first mode and controlling said training system to train said 
configurable network with all of said input vanables and said output variables in said 
dataset, 

said central processor controlling said sensitivity processor to determine 
said sensit.vity values associated with each of said input variables and compare said 
determined sensitivity values with a predetermined threshold and select only the ones of 
said input variables from said dataset having a sensitivity value that exceeds said 
sensitivity threshold, this being defined as a selected dataset; and 
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30 said central processor operable to configure said network to receive only 

input variables from said selected dataset and control said training system to train said 
configurable network on said selected dataset to provide a stored learned representation 
of a plant with only the selected input variables. 

2. The system of claim 1 wherein said training algorithm comprises a 
backpropagation training algorithm and said non-linear predictive network comprises a 
neural network. 
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3. A non-linear predictive network having stored therein a representation of 
a plant, comprising: 

a dataset of training patterns representing the input variables and 
measured output variables available to train a network on in order to generate a model of 
the plant, with each of the training patterns having input values associated with said input 
variables and corresponding output values associated with said measured output 
variables; 

a sensitivity processor for determining the sensitivity of each of said 
output variables as a function of each of said input variables, the sensitivity being the 
effect of each of the input variables on a learned representation of said output variables 
which learned representation was learned as a function of all of the input variables and' 
wherein the combined effects of all input vanables are taken into account in Said learned 
representation of said output variables; 

a comparator for companng said determined sensitivities output by said 
15 sensitivity processor with predetermined cnteria and selecting only ones of said input 
vanables having associated therewith sensitivities that meet said predetermined criteria; 
a configurable network having: 

an input layer for receiving only said select input variables, 
an output layer for outputting a defined number of output 
20 variables; and 

a hidden layer for mapping said input layer to said output 
layer through a stored learned representation of the plant; and 
a training device for training said configurable network on said dataset 
ut.hz.ng only input values associated with said select input vanables, said training device 
2. training sa.d network in accordance with a predetermined training algonthm. 

4. The network training system of claim 3 wherein sa.d defined number of 
output vanables compnses the number of said output variables in said dataset. 
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5. The network training system of claim 3 wherein said comparator is a user- 
controlled comparator for outputting said determined sensitivities of said input variables 
on a display for viewing by a user and providing an input device for allowing said user to 
select ones of said input variables that meet said predetermined criteria, said 

5 predetermined criteria being user-defined. 

6. The network training system of claim 4 wherein said predetermined 
criteria is operable to rank said determined sensitivities of said input variables with 
respect to each of said output variables, with the ones of said input variables having the 
lowest rank not included within said select input variables. 

7. The network training system of claim 4 wherein said sensitivities are 
determined over the range of input values for each of said input variables that affect each 
of said output variables, with said predetermined criteria accounting for sensitivity as a 
function of range such that if said sensitivity meets said predetermined criteria within said 
range, it constitutes a selected input variable. 

8. The network training system of claim 3 wherein said comparator 
comprises: 

a threshold memoiy for storing predetermined sensitivity thresholds; and 
a comparator for comparing the sensitivity of each of said input variables 
5 to said predetermined sensitivity thresholds and selecting the ones of said input variables 
having associated sensitivities that exceed said predetermined thresholds. 
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9. 



A control network for predicting control inputs for input to a d.stnbutive 
control system, which distributive control system outputs controls for input to a plant, 
the control inputs associated with the input variables to the plant, comprising: 
a control model for storing a representation of the plant; 
5 a desired value input defining desired operating parameters for the plant; 

said control model operable to receive as inputs the control inputs output 
by the distributive control system and said desired value input, and predict control input 
values for input as updated control inputs to the distributive control system that are 
necessary to achieve said desired operating parameters associated with said desired value 
10 input, 

a memory for storing sensitivity modifiers for defining the effect of each 
of the input variables on the plant output; and 

a modifier circuit for modifying the value of the predicted control input 
values prior to input to the distributive control system in accordance with a 
predetermined function of said pre-stored sensitivity modifiers in said memory. 



15 



10. The control network of cl«im 9 wherein said control model comprises a 
neural network having an input layer and an output layer with a hidden layer for mapping 
said input layer to said output layer through said stored representat.on of the plant. 

1 1 The control network of claim 9 wherein said control model comprises: 
a predictive network having stored therein a representat.on of the plant 
and for receiving the control inputs output by the distributive control system and 
generating a predicted plant output; 
5 an error generator for determining the error between said desired value 

input and the predicted output of said predictive network; and 

an inverse predictive network substantially similar to said predictive 
network for receiving sa.d error value on the output thereof and propagating the error 
through the network in an inverse direction to the input thereof and changine the input 
10 values to minimize said error, the input of said inverse network outputtmg said predicted 
control inputs 
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12. The control network of claim 1 1 wherein said error is propagated through 
said inverse predictive network and the error minimized in accordance with a 
backpropagation-to-activation algorithm. 

13. The control network of claim 9 and further comprising an analysis system 
for determining said sensitivity modifiers and, after determination thereof, storing said 
sensitivity modifiers in said memory. 

14. The control network of claim 13 wherein said stored representation is a 
learned representation and said analysis system comprises: 

a dataset memory for storing a plurality of training patterns on which said 
control model was trained to learn said stored representation, said training patterns 
5 representing the input variables and measured output variables available to train said 
control model on in order to generate said learned representation, with each of said 
training patterns having input values associated with said input variables and 
corresponding output values associated with said associated output variables; and 
means for determining the effect of each of the input variables on a 
1 0 learned representation of said output variables, which learned representation of said 
output variables was learned as a function of all of the input variables, and wherein the 
combined effects of all input variables are taken into account in said learned 
representation of said output variables. 

15. The control network of claim 14 wherein there are o { output variables and 
j input variables Xj, wherein said means for determining is operable to calculate said 
sensitivity modifiers for the jth variable/ith output variable, combination as follows: 



Npats do ^ ^ 

r — 

k~= 1 **j 

where N pats is the number of training patterns in said memory and o Li is the 
ith output for the kth pattern 
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16. The control network of claim 14 wherein there are o, output variables and 
j input vanables x, wherein said means for determtntng is operable to calculate said 
sensitivity modfe for the jth vanable/ith output variable, combination as follows- 



Npats 



k = I :' d*j i 



where N pats is the number of training patterns in said memory and o Kl is the 
ith output for the kth pattern. 

1 7. The control network of claim 14 wherein there are o ; output vanables and 
J input vanables x,, wherein sa.d means for determining is operable to calculate said 
sensttivity modifiers for the jth variable/ith output vanable, combination as follows 



( 1^1 



1. 2... Npats 



where N pjIS is the number of training patterns in said memory and o ti is the 
ith output for the kth pattern. 
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18. A system for defining the number of input variables of a system model, 
comprising the steps of: 

a dataset memory for storing a dataset of training patterns representing 
the measured input variables and measured output variables available to train the system 
5 model on in order to generate the system model, with each of said training patterns 
having input values associated with the measured input variables, and corresponding 
output values associated with the measured output variables; 

a sensitivity processor for determining the sensitivity of each of the output 
variables as a function of each of the input variables in said dataset of training patterns, 
10 the sensitivity being the effect of each of the measured input variables in the dataset of 
training patterns on a learned representation of the output variables, which learned 
representation was learned as a function of all of the measured input variables, and 
wherein the combined effects of all measured input variables are taken into account in the 
learned representation of the output variables, 
1 5 a comparator for comparing the sensitivities associated with each of the 

input variables with predetermined sensitivity criteria to determine the most sensitive 
input variables associated with the system model; and 

an output device for outputting the most sensitive input variables 
determined by said comparator. 

19. The system of Claim 18 wherein said output device comprises a rank 
ordering device for arranging said input variables in a rank order for output by said 
output device. 
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20. 
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A method f„, reducing ,he number of i„ pu , vanab , es „ f , ^ 
■ha, are operable to model a system, comprising the steps of: 

stonng in a dataset memory a dataset of training panems represe „ ting the 
measured inpu, vanables and measured output vanables avaiiable to train .he syslera 

input values associated w,«„ the measured inpu, vanables and correspond™ output 
values associated wtth the measured output variables, 

determiomgrhe sensitivity of each of the output varjab|es K , 
oach of the mpu, v anab , es ,„ lhe daIa!a of|ralnjng ^ ^ 

effec, of each of ,he measured mpu, vanabies in rhe datase, of training paHems on , 
■canted re„o„ of rhe output van*, wUch learned represe„,a,,o„ was , rarae d 
as a nd„c,,o„ofa,l of the measured ,„ pu , vannblea, and where ,„ fc ^ 
all measured mpu, variables are taken m,o aodoun, m ,he learned represent,,™ of the 
outpu, variables; and 

reducing the number of mpu, variables ,„ the system model hy 
determining the ones of the mpu, vanabies that mee, prede,ennr„ed sensttivny cntena 
these cc,,..u.ing „ e select mpu, vanables whach are input , 0 the system mode , ' 

21 Thcmethod of claim 20 wherein, he step of reducing the number of mpu, 
vanables comprises: P 

providing a configurable network that can be trained to lean, a 
representation of the system, 

oonngurtng the network to receive a,, of the input vanables and provide 
predtenve outputs corresponding to each of ,he output variables; 

tratning the network on al, of the measured inpu,' variables and measured 
outpu vanables shared in ,he memory to prov,de a learned represent of the ootpu, 
vanables in the step of determining; 

. ***■ <* ones of the mpu, vanables havmg sensitiv „ ies that mee , , he 
predetermined criteria; 

configuring the network to rece.ve only the select input vanables and 
tnurung the network on the dataset of training patterns stored in the 
memory by selecting o.y the measured values associated with the select input vanables 
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22. The method of claim 2 1 wherein the configurable network is a neural 
network which is operable to be trained in accordance with a backpropagation training 
algorithm. 

23 The method of claim 2 1 wherein the step of selecting the ones of the input 
variables having sensitivities that meet the predetermined criteria comprises comparing 
the sensitivities of all of the input variables to predetermined sensitivity thresholds and" 
selecting only the ones of the input variables having sensitivities that exceed the 
5 sensitivity thresholds. 

24. The method of claim 20 wherein the predetermined criteria are user- 
defined criteria and further compns.ng the step of displaying the sensirivity information 
and manually selecting the ones of the input variables having a sensitivity that meets the 
user-defined criteria. 

25. The method of claim 20 wherein the predetermined criteria is operable to 
rank the determined sensitivities of the h v ut variables with respect to each of the output 
variables, with the ones of the input variables having the lowest rank not included within 
the select ones of the input variables. 
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26 A method for generating control inputs with a distributed control system 
that generates control inputs for a plant, comprising the steps of: 

providing a control model that contains a representation of a plant; 
inputting a desired input value that defines desired operating parameters 

for the plant; 

inputting the control inputs generated by the distributed control system to 
the control model; 

predicting control input values that will allow the plant to operate at the 
des,red input value by processing the desired input value and current control inputs from 
the d 1S tnbuted control system to generate an error and minimizing the error with the 
control model, the pred.cted control inputs operable to be input to the distributed contro. 
system ,n order for the distributed control system generate new control inputs; 

stonng sensitivity modifiers for defining the effect of each of the input 
variables on the plant output; and 

modifying the value of the predicted control input values prior to input to 
the distnbuted control system in accordance with a predetermined function of the stored 
sensitivity modifiers. 



27. The method of claim 26 wherein the step of providing the control model 
compnses providing a neural network having an input layer and an output layer with a 
hidden layer for mapping the input layer to the output layer through the stored 
representation of the plant. 
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28. The method of claim 26 wherein the step of providing the control model 
comprises: 

providing a predictive network having stored therein a representation of 
the plant and for receiving the control inputs output by the distributive control system 
5 and predicting a plant output, 

providing an inverse predictive network substantially similar to the 
predictive network for receiving an error value on the output thereof and propagating the 
error through the inverse predictive network in an inverse direction to the input and 
changing the input values to minimize the error; 
10 the step of predicting the control inputs comprising: 

comparing the predicted plant output from the predictive 
network to the desired input value and generating the error for 
input to the output of the inverse predictive network, and 

controlling the predictive network and inverse network to 
minimize the error in the step of determining the error by 
alternately processing the input values from the inverse predictive 
network through the predictive network to generate a new error 
and then reprocess the error through the inverse network until the 
error is minimized, the value output from the input of the inverse 
predictive network comprising the predicted control inputs. 

29. The method of claim 28 wherein the error is propagated through the 
inverse predictive network and the error minimized in accordance with a 
backpropagation-to-activation algorithm. 

30. The method of claim 26 and further comprising determining the sensitivity 
modifiers and, after termination thereof, storing the sensitivities in the step of storing. 
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31. 



The method of claim 30 where.n the stored representation of the control 
mode. ,s a leaded represents and the step of determinmg the sensit.vity modifiers 
comprises. 

storing a Plurality of training patterns in a dataset memory ™,h the 
> .ratnang patterns comprising p„ teras on cvhtch the control mode, was tratned to lean, the 
stored representor,, the training pauems representing the i„ put ™ lbles and n<s 
output vanahles availahl. ro train the control mode, i„ order t0 generaK ^ 
representation, with each of .he .raining patterns havmg inpu, v, lues associared with ihe 
tnpu, vanahles and corresponding output v„ues associated wirh the associated output 
10 variables; and 

detenrurung the effect of each of the input variables on a learned 
representauon on the output vanables, wh.ch learned representat.on of the output 
vanables was learned as a funct.on of all of the input vanables, and wherein the 
combmed effects of all input variables are taken into account in the learned 
1 5 representation of the output variables. 



32. The method of claim 3 1 wherein there 
variables x,, wherein the step of detennining is operable to calculate the 
movers for the jth vanable/uh output vanable combinarion as follow 



are o { output variables and j input 
sensitivity 



t = i 



where is the number of training patterns in the dataset memory and 
o ti is the ith output for the kth pattern. 
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33. The method of claim 3 1 wherein there are o, output variables and j input 
variables x,-, wherein step of determining is operable to calculate the sensitivity modifiers 
for the jth variable/ith output variable combination as follows: 



Npats 



1 : 



where N pals is the number of training patterns in the dataset memory and 
5 o u is the ith output for the kth pattern. 

34. The method of claim 3 1 wherein there are o, output variables and j input 
variables ^ wherein step of determining is operable to calculate the sensitivity modifiers 
for the jth variable/ith output variable combination as follows: 



max I ! I , jte \ y 2~..Npats 

' dx j 



where N pals is the number of training patterns in the dataset and o ki is the 
5 ith output for the kth pattern. 
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35 A method for defining the number of input vanables of a system model, 
comprising the steps of: 

storing in a dataset memory a dataset of training patterns representing the 
measured input vanables and measured output vanables available to train the system 
model on in order to generate the system model, with each of the training patterns bavin* 
input values associated with the measured input variables and corresponding output 
values associated with the measured output variables; 

determining the sensitivity of each of the output vanables as a function of 
each of the input vanables in the dataset of training patterns, the sensitivity being the 
effect of each of the measured input vanables in the dataset of training patterns on a 
learned representation of the output vanables, which learned representation was learned 
as a function of all of the measured input vanables, and wherein the combined effects of 
all measured input vanables are taken into account in the learned representat.on of the 
output variables; 

comparing the sensitivities assoc.ated with each of the input varaibles 
with predetermined sensitivity cntena to determine the most sensitive input variables 
associated with the system mode!; and 

outputting the most sensitive input variables determined in the 
determining step. 



36. The method of Claim 35 wherein the step of outputting comprises 
arranging the input variables in a rank order for output by the step of outputting. 
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